{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "2ecf2e51",
   "metadata": {},
   "source": [
    "# Exploring the Model Zoo\n",
    "\n",
    "This experience introduces you to the core components of the FiftyOne Zoo:\n",
    "- The **Dataset Zoo** for accessing and exploring public datasets\n",
    "- The **Model Zoo** for running pre-trained models on your data\n",
    "- Creating your **own remotely-sourced datasets** for reuse and collaboration\n",
    "\n",
    "Whether you're a researcher, engineer, or educator, these tools help streamline your computer vision workflows in FiftyOne.\n",
    "\n",
    "> 💡 Make sure to run `pip install fiftyone torch torchvision` before starting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#!pip install fiftyone\n",
    "#!pip install torch torchvision"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## FiftyOne Zoo: A Hub for Datasets and Models\n",
    "\n",
    "FiftyOne Zoo provides easy access to a vast collection of pre-built datasets and pre-trained models. This notebook will guide you through exploring and using these resources.\n",
    "\n",
    "### Key Components:\n",
    "\n",
    "* **Dataset Zoo:** Offers a wide range of computer vision datasets, ready for immediate use.\n",
    "* **Model Zoo:** Provides pre-trained models for various tasks, enabling quick experimentation and deployment.\n",
    "\n",
    "Let's dive in!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import fiftyone as fo\n",
    "import fiftyone.zoo as foz"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model Zoo\n",
    "\n",
    "## Exploring the Model Zoo\n",
    "\n",
    "The Model Zoo provides pre-trained models that can be used for inference and evaluation.\n",
    "\n",
    "### Listing Available Models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"\\nAvailable Models:\")\n",
    "for model_name in foz.list_zoo_models():\n",
    "    print(f\"- {model_name}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "FiftyOne supports a wide range of models for classification, detection, and segmentation, please visit the [Model Zoo](https://docs.voxel51.com/model_zoo/models.html) for the full, up-to-date catalog.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loading a Model (Example: AlexNet on ImageNet)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading model from 'https://download.pytorch.org/models/alexnet-owt-4df8aa71.pth'...\n",
      " 100% |██████|    1.8Gb/1.8Gb [3.7s elapsed, 0s remaining, 527.5Mb/s]      \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Downloading: \"https://download.pytorch.org/models/alexnet-owt-7be5be79.pth\" to /home/paula/.cache/torch/hub/checkpoints/alexnet-owt-7be5be79.pth\n",
      "100.0%\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<fiftyone.zoo.models.torch.TorchvisionImageModel object at 0x7798cad07340>\n"
     ]
    }
   ],
   "source": [
    "model = foz.load_zoo_model(\"alexnet-imagenet-torch\")\n",
    "print(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![imagenet](https://cdn.voxel51.com/getting_started_model_dataset_zoo/notebook2/imagenet.webp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### Applying a Model to a Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dataset already downloaded\n",
      "Loading existing dataset 'imagenet-sample'. To reload from disk, either delete the existing dataset or provide a custom `dataset_name` to use\n",
      " 100% |███████████████| 1000/1000 [4.6s elapsed, 0s remaining, 277.2 samples/s]      \n",
      "None\n",
      "Session launched. Run `session.show()` to open the App in a cell output.\n"
     ]
    }
   ],
   "source": [
    "try:\n",
    "    dataset = foz.load_zoo_dataset(\"imagenet-sample\")\n",
    "    predictions = dataset.apply_model(model, label_field=\"predictions\")\n",
    "    print(predictions)\n",
    "except:\n",
    "    print(\"imagenet-sample dataset is not available, please install it if needed.\")\n",
    "\n",
    "session = fo.launch_app(dataset, port=5152, auto=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evaluating Model Performance\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading voxel51/fiftyone-plugins...\n",
      "  329.3Mb [12.2s elapsed, ? remaining, 32.9Mb/s]   \n",
      "Skipping existing plugin '@voxel51/evaluation'\n"
     ]
    }
   ],
   "source": [
    "!fiftyone plugins download https://github.com/voxel51/fiftyone-plugins --plugin-names @voxel51/evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                                precision    recall  f1-score   support\n",
      "\n",
      "                  Afghan hound       1.00      1.00      1.00         1\n",
      "             African chameleon       0.33      1.00      0.50         1\n",
      "             African crocodile       1.00      1.00      1.00         1\n",
      "              African elephant       0.00      0.00      0.00         0\n",
      "                  African grey       1.00      1.00      1.00         1\n",
      "           African hunting dog       1.00      1.00      1.00         1\n",
      "                      Airedale       1.00      1.00      1.00         1\n",
      "American Staffordshire terrier       0.00      0.00      0.00         1\n",
      "            American alligator       0.00      0.00      0.00         1\n",
      "           American black bear       0.00      0.00      0.00         1\n",
      "            American chameleon       0.00      0.00      0.00         1\n",
      "                 American coot       0.00      0.00      0.00         1\n",
      "                American egret       0.00      0.00      0.00         1\n",
      "              American lobster       1.00      1.00      1.00         1\n",
      "                        Angora       0.33      1.00      0.50         1\n",
      "                   Appenzeller       0.00      0.00      0.00         1\n",
      "                 Arabian camel       0.25      1.00      0.40         1\n",
      "                    Arctic fox       0.50      1.00      0.67         1\n",
      "            Australian terrier       0.00      0.00      0.00         1\n",
      "                      Band Aid       0.00      0.00      0.00         1\n",
      "            Bedlington terrier       1.00      1.00      1.00         1\n",
      "          Bernese mountain dog       0.00      0.00      0.00         1\n",
      "              Blenheim spaniel       1.00      1.00      1.00         1\n",
      "                 Border collie       0.00      0.00      0.00         1\n",
      "                Border terrier       0.00      0.00      0.00         1\n",
      "                   Boston bull       1.00      1.00      1.00         1\n",
      "          Bouvier des Flandres       0.00      0.00      0.00         1\n",
      "             Brabancon griffon       1.00      1.00      1.00         1\n",
      "              Brittany spaniel       1.00      1.00      1.00         1\n",
      "                     CD player       0.00      0.00      0.00         0\n",
      "                      Cardigan       0.00      0.00      0.00         2\n",
      "      Chesapeake Bay retriever       0.00      0.00      0.00         1\n",
      "                     Chihuahua       0.00      0.00      0.00         1\n",
      "                Dandie Dinmont       0.00      0.00      0.00         1\n",
      "                      Doberman       0.50      1.00      0.67         1\n",
      "                    Dutch oven       0.00      0.00      0.00         1\n",
      "                  Egyptian cat       0.00      0.00      0.00         0\n",
      "                English setter       0.00      0.00      0.00         1\n",
      "              English springer       0.00      0.00      0.00         1\n",
      "                   EntleBucher       1.00      1.00      1.00         1\n",
      "                    Eskimo dog       0.00      0.00      0.00         2\n",
      "      European fire salamander       1.00      1.00      1.00         1\n",
      "            European gallinule       0.00      0.00      0.00         1\n",
      "                French bulldog       1.00      1.00      1.00         1\n",
      "                   French horn       1.00      1.00      1.00         1\n",
      "               German shepherd       1.00      1.00      1.00         1\n",
      "   German short-haired pointer       1.00      1.00      1.00         1\n",
      "                  Gila monster       1.00      1.00      1.00         1\n",
      "                 Gordon setter       0.00      0.00      0.00         1\n",
      "                  Granny Smith       1.00      1.00      1.00         1\n",
      "                    Great Dane       0.00      0.00      0.00         1\n",
      "                Great Pyrenees       1.00      0.33      0.50         3\n",
      "                  Ibizan hound       0.00      0.00      0.00         1\n",
      "                  Indian cobra       1.00      1.00      1.00         1\n",
      "               Indian elephant       0.00      0.00      0.00         0\n",
      "                  Irish setter       1.00      1.00      1.00         1\n",
      "                 Irish terrier       1.00      1.00      1.00         1\n",
      "           Irish water spaniel       0.50      1.00      0.67         1\n",
      "               Irish wolfhound       0.00      0.00      0.00         1\n",
      "             Italian greyhound       1.00      1.00      1.00         2\n",
      "              Japanese spaniel       1.00      1.00      1.00         1\n",
      "            Kerry blue terrier       0.50      1.00      0.67         1\n",
      "                 Komodo dragon       0.50      1.00      0.67         1\n",
      "              Lakeland terrier       1.00      1.00      1.00         1\n",
      "                      Leonberg       1.00      1.00      1.00         1\n",
      "                         Lhasa       1.00      1.00      1.00         1\n",
      "                        Loafer       1.00      1.00      1.00         1\n",
      "                Madagascar cat       0.50      1.00      0.67         1\n",
      "                   Maltese dog       0.50      1.00      0.67         1\n",
      "              Mexican hairless       0.00      0.00      0.00         0\n",
      "                       Model T       0.50      1.00      0.67         1\n",
      "                  Newfoundland       1.00      1.00      1.00         1\n",
      "               Norfolk terrier       1.00      1.00      1.00         1\n",
      "            Norwegian elkhound       1.00      1.00      1.00         1\n",
      "               Norwich terrier       0.00      0.00      0.00         1\n",
      "          Old English sheepdog       0.00      0.00      0.00         1\n",
      "                      Pekinese       1.00      1.00      1.00         1\n",
      "                      Pembroke       0.00      0.00      0.00         0\n",
      "                   Persian cat       1.00      1.00      1.00         1\n",
      "                    Petri dish       0.50      1.00      0.67         1\n",
      "               Polaroid camera       0.50      0.50      0.50         2\n",
      "                    Pomeranian       1.00      1.00      1.00         1\n",
      "           Rhodesian ridgeback       0.00      0.00      0.00         1\n",
      "                    Rottweiler       0.00      0.00      0.00         2\n",
      "                 Saint Bernard       1.00      1.00      1.00         1\n",
      "                        Saluki       0.50      0.50      0.50         2\n",
      "                       Samoyed       0.50      1.00      0.67         1\n",
      "                Scotch terrier       1.00      1.00      1.00         1\n",
      "            Scottish deerhound       0.50      1.00      0.67         1\n",
      "              Sealyham terrier       0.00      0.00      0.00         1\n",
      "             Shetland sheepdog       0.00      0.00      0.00         1\n",
      "                      Shih-Tzu       0.50      0.50      0.50         2\n",
      "                   Siamese cat       0.50      1.00      0.67         1\n",
      "                Siberian husky       0.33      1.00      0.50         1\n",
      "     Staffordshire bullterrier       1.00      1.00      1.00         1\n",
      "                Sussex spaniel       0.00      0.00      0.00         1\n",
      "               Tibetan terrier       0.00      0.00      0.00         1\n",
      "                  Walker hound       0.00      0.00      0.00         2\n",
      "                    Weimaraner       0.50      1.00      0.67         1\n",
      "        Welsh springer spaniel       0.50      1.00      0.67         1\n",
      "   West Highland white terrier       1.00      1.00      1.00         1\n",
      "             Yorkshire terrier       0.00      0.00      0.00         1\n",
      "                         abaya       0.00      0.00      0.00         0\n",
      "                     accordion       0.00      0.00      0.00         1\n",
      "                         acorn       1.00      1.00      1.00         1\n",
      "                  acorn squash       0.50      1.00      0.67         1\n",
      "               acoustic guitar       0.00      0.00      0.00         1\n",
      "                       admiral       0.00      0.00      0.00         1\n",
      "                 affenpinscher       1.00      1.00      1.00         1\n",
      "                         agama       0.00      0.00      0.00         0\n",
      "                        agaric       1.00      1.00      1.00         2\n",
      "              aircraft carrier       0.00      0.00      0.00         2\n",
      "                      airliner       1.00      1.00      1.00         1\n",
      "                       airship       0.00      0.00      0.00         2\n",
      "                     albatross       0.00      0.00      0.00         1\n",
      "              alligator lizard       0.00      0.00      0.00         1\n",
      "                         altar       0.00      0.00      0.00         1\n",
      "                     ambulance       0.50      1.00      0.67         1\n",
      "                  anemone fish       1.00      0.50      0.67         2\n",
      "                           ant       0.00      0.00      0.00         1\n",
      "                        apiary       0.50      1.00      0.67         1\n",
      "                         apron       1.00      1.00      1.00         1\n",
      "                     armadillo       1.00      1.00      1.00         1\n",
      "                     artichoke       0.00      0.00      0.00         0\n",
      "                        ashcan       0.00      0.00      0.00         1\n",
      "                       axolotl       0.50      1.00      0.67         1\n",
      "                        baboon       0.00      0.00      0.00         1\n",
      "                      backpack       0.00      0.00      0.00         0\n",
      "                        bakery       1.00      1.00      1.00         1\n",
      "                  balance beam       0.00      0.00      0.00         1\n",
      "                    bald eagle       0.33      0.50      0.40         2\n",
      "                       balloon       1.00      1.00      1.00         1\n",
      "                    ballplayer       0.00      0.00      0.00         1\n",
      "                     ballpoint       0.50      1.00      0.67         1\n",
      "                        banana       0.00      0.00      0.00         0\n",
      "                  banded gecko       0.00      0.00      0.00         1\n",
      "                         banjo       1.00      1.00      1.00         1\n",
      "                     bannister       0.00      0.00      0.00         0\n",
      "                       barbell       1.00      1.00      1.00         1\n",
      "                  barber chair       0.50      1.00      0.67         1\n",
      "                    barbershop       0.00      0.00      0.00         1\n",
      "                          barn       0.50      1.00      0.67         1\n",
      "                   barn spider       1.00      1.00      1.00         1\n",
      "                     barometer       1.00      1.00      1.00         1\n",
      "                    barracouta       1.00      1.00      1.00         1\n",
      "                        barrel       1.00      1.00      1.00         1\n",
      "                        barrow       0.00      0.00      0.00         1\n",
      "                      baseball       1.00      1.00      1.00         1\n",
      "                       basenji       0.00      0.00      0.00         1\n",
      "                    basketball       1.00      1.00      1.00         1\n",
      "                        basset       0.00      0.00      0.00         2\n",
      "                      bassinet       0.00      0.00      0.00         1\n",
      "                       bassoon       0.00      0.00      0.00         1\n",
      "                    bath towel       0.00      0.00      0.00         0\n",
      "                   bathing cap       0.00      0.00      0.00         1\n",
      "                       bathtub       0.00      0.00      0.00         2\n",
      "                   beach wagon       0.50      0.50      0.50         2\n",
      "                        beacon       0.00      0.00      0.00         1\n",
      "                        beagle       0.67      0.67      0.67         3\n",
      "                        beaker       0.00      0.00      0.00         1\n",
      "                      bearskin       1.00      1.00      1.00         1\n",
      "                        beaver       0.50      1.00      0.67         1\n",
      "                           bee       1.00      0.50      0.67         2\n",
      "                     bee eater       0.00      0.00      0.00         0\n",
      "                   beer bottle       1.00      1.00      1.00         1\n",
      "                    beer glass       1.00      1.00      1.00         1\n",
      "                     bell cote       0.00      0.00      0.00         1\n",
      "                   bell pepper       1.00      1.00      1.00         1\n",
      "                           bib       0.00      0.00      0.00         1\n",
      "         bicycle-built-for-two       1.00      0.50      0.67         2\n",
      "                       bighorn       1.00      1.00      1.00         1\n",
      "                        bikini       1.00      1.00      1.00         1\n",
      "                        binder       0.50      1.00      0.67         1\n",
      "                    binoculars       0.00      0.00      0.00         1\n",
      "                     birdhouse       1.00      1.00      1.00         1\n",
      "                         bison       0.50      1.00      0.67         1\n",
      "                       bittern       0.00      0.00      0.00         1\n",
      "  black and gold garden spider       1.00      1.00      1.00         1\n",
      "                  black grouse       1.00      1.00      1.00         1\n",
      "                   black stork       0.00      0.00      0.00         1\n",
      "                    black swan       0.50      1.00      0.67         1\n",
      "                   black widow       1.00      1.00      1.00         1\n",
      "           black-footed ferret       1.00      1.00      1.00         1\n",
      "                      bluetick       0.33      1.00      0.50         1\n",
      "               boa constrictor       0.00      0.00      0.00         1\n",
      "                     boathouse       0.25      1.00      0.40         1\n",
      "                       bobsled       0.00      0.00      0.00         2\n",
      "                        bolete       1.00      1.00      1.00         1\n",
      "                      bolo tie       0.00      0.00      0.00         1\n",
      "                        bonnet       0.00      0.00      0.00         2\n",
      "                   book jacket       0.00      0.00      0.00         1\n",
      "                      bookcase       1.00      1.00      1.00         1\n",
      "                      bookshop       0.00      0.00      0.00         1\n",
      "                        borzoi       0.00      0.00      0.00         1\n",
      "                     bottlecap       1.00      1.00      1.00         1\n",
      "                           bow       1.00      1.00      1.00         1\n",
      "                       bow tie       0.00      0.00      0.00         0\n",
      "                         boxer       0.50      1.00      0.67         1\n",
      "                   brain coral       1.00      1.00      1.00         1\n",
      "                     brambling       0.00      0.00      0.00         1\n",
      "                         brass       0.33      1.00      0.50         1\n",
      "                     brassiere       0.00      0.00      0.00         0\n",
      "                    breakwater       0.00      0.00      0.00         2\n",
      "                   breastplate       0.00      0.00      0.00         2\n",
      "                        briard       0.50      0.50      0.50         2\n",
      "                      broccoli       1.00      1.00      1.00         1\n",
      "                         broom       0.50      1.00      0.67         2\n",
      "                    brown bear       1.00      1.00      1.00         1\n",
      "                        bubble       1.00      1.00      1.00         1\n",
      "                        bucket       0.00      0.00      0.00         0\n",
      "                       buckeye       1.00      1.00      1.00         1\n",
      "                        buckle       0.50      1.00      0.67         1\n",
      "                        bulbul       0.00      0.00      0.00         1\n",
      "                  bull mastiff       0.50      1.00      0.67         1\n",
      "                  bullet train       0.50      0.50      0.50         2\n",
      "              bulletproof vest       0.50      1.00      0.67         1\n",
      "                      bullfrog       0.00      0.00      0.00         1\n",
      "                       burrito       0.50      1.00      0.67         1\n",
      "                       bustard       0.00      0.00      0.00         1\n",
      "                  butcher shop       1.00      1.00      1.00         1\n",
      "              butternut squash       0.00      0.00      0.00         1\n",
      "                           cab       0.00      0.00      0.00         2\n",
      "             cabbage butterfly       0.50      1.00      0.67         1\n",
      "                         cairn       0.50      1.00      0.67         1\n",
      "                       caldron       0.00      0.00      0.00         1\n",
      "                    can opener       1.00      1.00      1.00         1\n",
      "                        candle       0.00      0.00      0.00         0\n",
      "                        cannon       0.00      0.00      0.00         0\n",
      "                         canoe       1.00      1.00      1.00         1\n",
      "                      capuchin       0.00      0.00      0.00         1\n",
      "                    car mirror       1.00      1.00      1.00         1\n",
      "                     car wheel       0.00      0.00      0.00         1\n",
      "                     carbonara       0.00      0.00      0.00         1\n",
      "                      cardigan       1.00      0.50      0.67         2\n",
      "                       cardoon       1.00      0.50      0.67         2\n",
      "                      carousel       0.00      0.00      0.00         1\n",
      "               carpenter's kit       0.00      0.00      0.00         1\n",
      "                        carton       0.00      0.00      0.00         0\n",
      "                  cash machine       0.50      1.00      0.67         1\n",
      "                      cassette       1.00      1.00      1.00         1\n",
      "               cassette player       0.00      0.00      0.00         1\n",
      "                        castle       1.00      1.00      1.00         1\n",
      "                     catamaran       1.00      1.00      1.00         1\n",
      "                   cauliflower       1.00      1.00      1.00         1\n",
      "                         cello       1.00      1.00      1.00         1\n",
      "            cellular telephone       0.50      0.50      0.50         2\n",
      "                         chain       1.00      1.00      1.00         1\n",
      "                    chain mail       0.50      1.00      0.67         1\n",
      "                     chain saw       0.00      0.00      0.00         1\n",
      "               chainlink fence       1.00      0.50      0.67         2\n",
      "            chambered nautilus       1.00      0.50      0.67         2\n",
      "                  cheeseburger       1.00      1.00      1.00         1\n",
      "                       cheetah       1.00      1.00      1.00         2\n",
      "                         chest       0.00      0.00      0.00         0\n",
      "                     chickadee       0.00      0.00      0.00         1\n",
      "                    chiffonier       0.00      0.00      0.00         1\n",
      "                         chime       0.00      0.00      0.00         2\n",
      "                    chimpanzee       1.00      1.00      1.00         1\n",
      "                 china cabinet       1.00      0.50      0.67         2\n",
      "                        chiton       0.50      1.00      0.67         1\n",
      "                          chow       1.00      1.00      1.00         1\n",
      "                        church       0.50      1.00      0.67         1\n",
      "                        cicada       1.00      1.00      1.00         1\n",
      "                        cinema       1.00      1.00      1.00         1\n",
      "                       cleaver       0.00      0.00      0.00         1\n",
      "                cliff dwelling       1.00      1.00      1.00         1\n",
      "                         cloak       0.00      0.00      0.00         0\n",
      "                          clog       0.00      0.00      0.00         0\n",
      "                       clumber       1.00      1.00      1.00         1\n",
      "                cocker spaniel       0.00      0.00      0.00         1\n",
      "                     cockroach       0.50      1.00      0.67         1\n",
      "                     coffeepot       0.00      0.00      0.00         1\n",
      "                          coho       0.50      1.00      0.67         1\n",
      "                          coil       0.00      0.00      0.00         0\n",
      "                        collie       0.00      0.00      0.00         1\n",
      "                    comic book       0.00      0.00      0.00         0\n",
      "                 common iguana       1.00      1.00      1.00         1\n",
      "                   common newt       0.00      0.00      0.00         1\n",
      "             computer keyboard       0.00      0.00      0.00         1\n",
      "                         conch       0.50      1.00      0.67         1\n",
      "                 confectionery       1.00      1.00      1.00         1\n",
      "                      consomme       0.00      0.00      0.00         1\n",
      "                container ship       1.00      1.00      1.00         1\n",
      "                   convertible       0.00      0.00      0.00         1\n",
      "                  coral fungus       1.00      1.00      1.00         1\n",
      "                    coral reef       1.00      0.50      0.67         2\n",
      "                     corkscrew       1.00      1.00      1.00         1\n",
      "                          corn       0.00      0.00      0.00         1\n",
      "                        cornet       0.50      1.00      0.67         1\n",
      "                        coucal       1.00      1.00      1.00         1\n",
      "                        cougar       1.00      1.00      1.00         1\n",
      "                   cowboy boot       0.00      0.00      0.00         0\n",
      "                    cowboy hat       0.00      0.00      0.00         1\n",
      "                        coyote       0.00      0.00      0.00         0\n",
      "                        cradle       1.00      1.00      1.00         1\n",
      "                         crane       0.67      0.67      0.67         3\n",
      "                  crash helmet       0.00      0.00      0.00         2\n",
      "                         crate       0.00      0.00      0.00         2\n",
      "                      crayfish       0.00      0.00      0.00         1\n",
      "                          crib       1.00      1.00      1.00         1\n",
      "                       cricket       0.00      0.00      0.00         1\n",
      "                  croquet ball       0.00      0.00      0.00         1\n",
      "              crossword puzzle       1.00      1.00      1.00         1\n",
      "                        crutch       0.00      0.00      0.00         0\n",
      "                      cucumber       0.00      0.00      0.00         1\n",
      "                       cuirass       0.00      0.00      0.00         0\n",
      "                           cup       0.00      0.00      0.00         1\n",
      "        curly-coated retriever       0.00      0.00      0.00         1\n",
      "                 custard apple       1.00      1.00      1.00         1\n",
      "                         daisy       0.00      0.00      0.00         1\n",
      "                     dalmatian       1.00      1.00      1.00         1\n",
      "                           dam       0.00      0.00      0.00         1\n",
      "                     damselfly       1.00      1.00      1.00         1\n",
      "                          desk       1.00      0.67      0.80         3\n",
      "              desktop computer       0.00      0.00      0.00         1\n",
      "                         dhole       0.00      0.00      0.00         1\n",
      "                dial telephone       1.00      1.00      1.00         1\n",
      "                   diamondback       0.00      0.00      0.00         1\n",
      "                        diaper       0.33      1.00      0.50         1\n",
      "                 digital clock       0.00      0.00      0.00         1\n",
      "                 digital watch       1.00      0.50      0.67         2\n",
      "                  dining table       1.00      1.00      1.00         1\n",
      "                       dishrag       0.50      1.00      0.67         1\n",
      "                    dishwasher       0.67      1.00      0.80         2\n",
      "                    disk brake       1.00      1.00      1.00         1\n",
      "                          dock       0.00      0.00      0.00         0\n",
      "                       dogsled       1.00      1.00      1.00         1\n",
      "                          dome       0.00      0.00      0.00         2\n",
      "                         dough       1.00      1.00      1.00         1\n",
      "                     dowitcher       0.33      1.00      0.50         1\n",
      "                     dragonfly       0.00      0.00      0.00         1\n",
      "                         drake       1.00      1.00      1.00         1\n",
      "             drilling platform       0.00      0.00      0.00         0\n",
      "                          drum       0.00      0.00      0.00         1\n",
      "                     drumstick       0.00      0.00      0.00         1\n",
      "                      dumbbell       1.00      0.50      0.67         2\n",
      "                   dung beetle       1.00      1.00      1.00         1\n",
      "                           ear       1.00      1.00      1.00         1\n",
      "                     earthstar       1.00      1.00      1.00         1\n",
      "                       echidna       0.50      1.00      0.67         1\n",
      "                           eel       0.50      1.00      0.67         1\n",
      "                           eft       0.50      1.00      0.67         1\n",
      "                        eggnog       0.00      0.00      0.00         1\n",
      "                  electric fan       0.00      0.00      0.00         1\n",
      "               electric guitar       1.00      1.00      1.00         1\n",
      "           electric locomotive       1.00      1.00      1.00         1\n",
      "                  electric ray       0.50      1.00      0.67         1\n",
      "          entertainment center       1.00      0.50      0.67         2\n",
      "                      envelope       0.00      0.00      0.00         0\n",
      "                      espresso       1.00      1.00      1.00         1\n",
      "                   face powder       0.00      0.00      0.00         1\n",
      "                   feather boa       1.00      1.00      1.00         1\n",
      "                  fiddler crab       0.00      0.00      0.00         2\n",
      "                           fig       1.00      0.50      0.67         2\n",
      "                          file       1.00      0.50      0.67         2\n",
      "                   fire engine       1.00      1.00      1.00         1\n",
      "                   fire screen       0.00      0.00      0.00         1\n",
      "                      fireboat       1.00      1.00      1.00         1\n",
      "                      flagpole       0.00      0.00      0.00         0\n",
      "                      flamingo       1.00      1.00      1.00         1\n",
      "         flat-coated retriever       0.33      1.00      0.50         1\n",
      "                      flatworm       1.00      1.00      1.00         1\n",
      "                         flute       0.00      0.00      0.00         1\n",
      "               football helmet       0.00      0.00      0.00         1\n",
      "                      forklift       0.00      0.00      0.00         1\n",
      "                      fountain       0.50      0.50      0.50         2\n",
      "                  fountain pen       1.00      1.00      1.00         1\n",
      "                   four-poster       0.50      1.00      0.67         1\n",
      "                  fox squirrel       0.00      0.00      0.00         0\n",
      "                   freight car       1.00      1.00      1.00         1\n",
      "                frilled lizard       0.00      0.00      0.00         1\n",
      "                    frying pan       0.00      0.00      0.00         1\n",
      "                      fur coat       1.00      1.00      1.00         1\n",
      "                 garbage truck       1.00      1.00      1.00         1\n",
      "                 garden spider       0.00      0.00      0.00         0\n",
      "                  garter snake       0.33      0.50      0.40         2\n",
      "                      gas pump       1.00      1.00      1.00         1\n",
      "                       gasmask       0.00      0.00      0.00         1\n",
      "                       gazelle       0.00      0.00      0.00         1\n",
      "                        geyser       1.00      1.00      1.00         1\n",
      "                   giant panda       1.00      0.50      0.67         2\n",
      "               giant schnauzer       1.00      1.00      1.00         1\n",
      "                        gibbon       1.00      0.50      0.67         2\n",
      "                       go-kart       0.00      0.00      0.00         1\n",
      "                        goblet       1.00      0.50      0.67         2\n",
      "                     goldfinch       1.00      1.00      1.00         1\n",
      "                      goldfish       0.00      0.00      0.00         1\n",
      "                      golfcart       0.00      0.00      0.00         1\n",
      "                       gondola       0.50      1.00      0.67         1\n",
      "                          gong       0.00      0.00      0.00         0\n",
      "                         goose       1.00      1.00      1.00         1\n",
      "                       gorilla       0.00      0.00      0.00         1\n",
      "                          gown       0.00      0.00      0.00         0\n",
      "                   grand piano       0.33      1.00      0.50         1\n",
      "                   grasshopper       0.33      1.00      0.50         1\n",
      "                great grey owl       1.00      1.00      1.00         1\n",
      "             great white shark       1.00      1.00      1.00         1\n",
      "                  green lizard       0.00      0.00      0.00         1\n",
      "                   green mamba       0.00      0.00      0.00         2\n",
      "                   green snake       0.00      0.00      0.00         0\n",
      "                    greenhouse       0.00      0.00      0.00         0\n",
      "                    grey whale       1.00      1.00      1.00         2\n",
      "                 grocery store       1.00      0.50      0.67         2\n",
      "                   groenendael       0.00      0.00      0.00         1\n",
      "                         groom       1.00      0.50      0.67         2\n",
      "                 ground beetle       0.00      0.00      0.00         1\n",
      "                     guacamole       0.00      0.00      0.00         1\n",
      "                    guillotine       1.00      0.50      0.67         2\n",
      "                    guinea pig       0.50      1.00      0.67         1\n",
      "                     gyromitra       1.00      0.50      0.67         2\n",
      "                    hair slide       0.00      0.00      0.00         2\n",
      "                    half track       0.00      0.00      0.00         1\n",
      "                    hammerhead       1.00      1.00      1.00         1\n",
      "                       hamster       0.00      0.00      0.00         2\n",
      "                   hand blower       0.50      1.00      0.67         1\n",
      "                  handkerchief       0.50      1.00      0.67         1\n",
      "                     hard disc       0.00      0.00      0.00         1\n",
      "                          hare       1.00      1.00      1.00         2\n",
      "                     harmonica       0.00      0.00      0.00         1\n",
      "                          harp       0.00      0.00      0.00         1\n",
      "                    hartebeest       0.00      0.00      0.00         0\n",
      "                     harvester       1.00      1.00      1.00         2\n",
      "                    harvestman       0.50      0.50      0.50         2\n",
      "                       hatchet       0.00      0.00      0.00         0\n",
      "                  head cabbage       0.50      1.00      0.67         1\n",
      "                           hen       1.00      1.00      1.00         2\n",
      "              hen-of-the-woods       0.00      0.00      0.00         0\n",
      "                   hermit crab       1.00      1.00      1.00         1\n",
      "                           hip       0.50      1.00      0.67         1\n",
      "                  hippopotamus       1.00      1.00      1.00         1\n",
      "                 hognose snake       0.50      1.00      0.67         1\n",
      "                       holster       0.33      1.00      0.50         1\n",
      "                  home theater       0.50      1.00      0.67         1\n",
      "                     honeycomb       0.00      0.00      0.00         0\n",
      "                          hook       0.00      0.00      0.00         2\n",
      "                     hoopskirt       1.00      1.00      1.00         1\n",
      "                horizontal bar       0.67      1.00      0.80         2\n",
      "                  horned viper       0.00      0.00      0.00         1\n",
      "                    horse cart       1.00      1.00      1.00         1\n",
      "                       hot pot       1.00      0.75      0.86         4\n",
      "                        hotdog       0.00      0.00      0.00         2\n",
      "                     hourglass       0.00      0.00      0.00         1\n",
      "                   house finch       1.00      1.00      1.00         1\n",
      "                 howler monkey       0.50      0.50      0.50         2\n",
      "                   hummingbird       0.50      1.00      0.67         1\n",
      "                         hyena       0.00      0.00      0.00         1\n",
      "                          iPod       0.50      1.00      0.67         1\n",
      "                          ibex       0.00      0.00      0.00         1\n",
      "                      ice bear       0.00      0.00      0.00         1\n",
      "                     ice cream       0.00      0.00      0.00         1\n",
      "                     ice lolly       0.00      0.00      0.00         1\n",
      "                        impala       1.00      1.00      1.00         1\n",
      "                indigo bunting       0.00      0.00      0.00         1\n",
      "                         indri       1.00      1.00      1.00         1\n",
      "                          iron       0.00      0.00      0.00         0\n",
      "                        isopod       0.00      0.00      0.00         1\n",
      "                       jacamar       1.00      1.00      1.00         2\n",
      "               jack-o'-lantern       1.00      1.00      1.00         1\n",
      "                     jackfruit       1.00      1.00      1.00         1\n",
      "                        jaguar       1.00      1.00      1.00         1\n",
      "                           jay       0.00      0.00      0.00         1\n",
      "                          jean       1.00      1.00      1.00         1\n",
      "                          jeep       1.00      1.00      1.00         1\n",
      "                     jellyfish       1.00      1.00      1.00         1\n",
      "                        jersey       1.00      1.00      1.00         1\n",
      "                 jigsaw puzzle       0.33      1.00      0.50         1\n",
      "                    jinrikisha       0.33      1.00      0.50         1\n",
      "                      joystick       0.00      0.00      0.00         1\n",
      "                         junco       1.00      1.00      1.00         1\n",
      "                      keeshond       0.50      1.00      0.67         1\n",
      "                  killer whale       1.00      1.00      1.00         1\n",
      "                        kimono       0.00      0.00      0.00         1\n",
      "                     king crab       0.50      1.00      0.67         1\n",
      "                  king penguin       1.00      1.00      1.00         1\n",
      "                    king snake       1.00      1.00      1.00         1\n",
      "                       kit fox       0.50      1.00      0.67         1\n",
      "                          kite       0.50      0.50      0.50         2\n",
      "                      knee pad       1.00      1.00      1.00         1\n",
      "                          knot       1.00      1.00      1.00         1\n",
      "                         koala       1.00      1.00      1.00         1\n",
      "                      komondor       0.50      1.00      0.67         1\n",
      "                        kuvasz       0.00      0.00      0.00         0\n",
      "                      lab coat       1.00      1.00      1.00         1\n",
      "                      lacewing       0.00      0.00      0.00         2\n",
      "                         ladle       0.00      0.00      0.00         2\n",
      "                       ladybug       0.00      0.00      0.00         1\n",
      "                      lakeside       0.00      0.00      0.00         1\n",
      "                     lampshade       0.00      0.00      0.00         1\n",
      "                        langur       0.00      0.00      0.00         1\n",
      "                        laptop       0.00      0.00      0.00         0\n",
      "                    lawn mower       1.00      1.00      1.00         1\n",
      "                   leaf beetle       0.25      1.00      0.40         1\n",
      "                    leafhopper       1.00      1.00      1.00         1\n",
      "                         lemon       0.50      1.00      0.67         1\n",
      "                      lens cap       0.00      0.00      0.00         2\n",
      "                       leopard       0.00      0.00      0.00         0\n",
      "                  lesser panda       0.00      0.00      0.00         0\n",
      "                 letter opener       0.00      0.00      0.00         2\n",
      "                       library       0.00      0.00      0.00         1\n",
      "                      lifeboat       1.00      1.00      1.00         1\n",
      "                       lighter       0.33      1.00      0.50         1\n",
      "                     limousine       1.00      1.00      1.00         1\n",
      "                       limpkin       0.00      0.00      0.00         1\n",
      "                         liner       1.00      0.67      0.80         3\n",
      "                          lion       0.50      1.00      0.67         1\n",
      "                      lionfish       0.50      1.00      0.67         1\n",
      "                      lipstick       0.33      0.50      0.40         2\n",
      "                         llama       1.00      1.00      1.00         1\n",
      "                    loggerhead       1.00      1.00      1.00         1\n",
      "            long-horned beetle       0.00      0.00      0.00         1\n",
      "                      lorikeet       1.00      1.00      1.00         1\n",
      "                   loudspeaker       0.00      0.00      0.00         0\n",
      "                    lumbermill       1.00      1.00      1.00         1\n",
      "                      lycaenid       0.00      0.00      0.00         1\n",
      "                          lynx       0.00      0.00      0.00         2\n",
      "                       macaque       0.00      0.00      0.00         0\n",
      "                         macaw       1.00      1.00      1.00         1\n",
      "              magnetic compass       0.00      0.00      0.00         1\n",
      "                        magpie       1.00      1.00      1.00         1\n",
      "                       mailbag       0.00      0.00      0.00         2\n",
      "                       mailbox       0.50      1.00      0.67         1\n",
      "                       maillot       0.33      1.00      0.50         1\n",
      "                      malamute       0.33      1.00      0.50         1\n",
      "                      malinois       0.67      1.00      0.80         2\n",
      "                 manhole cover       1.00      0.50      0.67         2\n",
      "                        mantis       0.00      0.00      0.00         1\n",
      "                        maraca       0.00      0.00      0.00         0\n",
      "                       marimba       0.00      0.00      0.00         1\n",
      "                      marmoset       1.00      1.00      1.00         1\n",
      "                        marmot       0.50      1.00      0.67         1\n",
      "                 mashed potato       1.00      1.00      1.00         1\n",
      "                          mask       0.50      1.00      0.67         1\n",
      "                    matchstick       0.00      0.00      0.00         1\n",
      "                       maypole       0.50      1.00      0.67         1\n",
      "                          maze       1.00      0.50      0.67         2\n",
      "                 measuring cup       1.00      1.00      1.00         1\n",
      "                     meat loaf       0.00      0.00      0.00         1\n",
      "                medicine chest       0.00      0.00      0.00         1\n",
      "                       meerkat       1.00      1.00      1.00         1\n",
      "                      megalith       1.00      1.00      1.00         1\n",
      "                          menu       1.00      1.00      1.00         1\n",
      "                    microphone       0.50      1.00      0.67         1\n",
      "                     microwave       1.00      1.00      1.00         1\n",
      "              military uniform       0.00      0.00      0.00         1\n",
      "                      milk can       0.00      0.00      0.00         1\n",
      "            miniature pinscher       1.00      1.00      1.00         2\n",
      "              miniature poodle       1.00      1.00      1.00         1\n",
      "           miniature schnauzer       1.00      1.00      1.00         1\n",
      "                       minibus       1.00      1.00      1.00         1\n",
      "                     miniskirt       0.00      0.00      0.00         2\n",
      "                       minivan       0.00      0.00      0.00         1\n",
      "                          mink       0.00      0.00      0.00         1\n",
      "                       missile       1.00      1.00      1.00         1\n",
      "                        mitten       0.00      0.00      0.00         0\n",
      "                   mixing bowl       0.00      0.00      0.00         3\n",
      "                   mobile home       0.25      1.00      0.40         1\n",
      "                         modem       0.00      0.00      0.00         0\n",
      "                       monarch       1.00      1.00      1.00         1\n",
      "                     monastery       1.00      1.00      1.00         1\n",
      "                      mongoose       0.00      0.00      0.00         1\n",
      "                       monitor       0.00      0.00      0.00         1\n",
      "                         moped       0.00      0.00      0.00         1\n",
      "                        mortar       0.50      1.00      0.67         1\n",
      "                   mortarboard       0.00      0.00      0.00         2\n",
      "                        mosque       1.00      1.00      1.00         1\n",
      "                  mosquito net       0.00      0.00      0.00         1\n",
      "                 motor scooter       0.00      0.00      0.00         1\n",
      "                 mountain bike       1.00      1.00      1.00         1\n",
      "                 mountain tent       1.00      1.00      1.00         1\n",
      "                         mouse       0.00      0.00      0.00         1\n",
      "                    moving van       0.00      0.00      0.00         1\n",
      "                    mud turtle       0.00      0.00      0.00         1\n",
      "                      mushroom       0.00      0.00      0.00         0\n",
      "                          nail       0.25      1.00      0.40         1\n",
      "                    neck brace       0.00      0.00      0.00         1\n",
      "                      necklace       1.00      0.50      0.67         2\n",
      "                      nematode       0.00      0.00      0.00         1\n",
      "                        nipple       0.00      0.00      0.00         1\n",
      "                      notebook       1.00      0.50      0.67         2\n",
      "                       obelisk       1.00      0.33      0.50         3\n",
      "                          oboe       1.00      1.00      1.00         1\n",
      "                       ocarina       0.00      0.00      0.00         1\n",
      "                      odometer       1.00      1.00      1.00         1\n",
      "                    oil filter       0.00      0.00      0.00         1\n",
      "                        orange       1.00      1.00      1.00         1\n",
      "                     orangutan       1.00      1.00      1.00         1\n",
      "                  oscilloscope       0.50      0.50      0.50         2\n",
      "                       ostrich       1.00      1.00      1.00         1\n",
      "                         otter       0.00      0.00      0.00         1\n",
      "                     overskirt       0.00      0.00      0.00         1\n",
      "                            ox       1.00      1.00      1.00         1\n",
      "                        oxcart       1.00      1.00      1.00         1\n",
      "                   oxygen mask       0.00      0.00      0.00         1\n",
      "                 oystercatcher       1.00      1.00      1.00         1\n",
      "                        packet       0.00      0.00      0.00         1\n",
      "                        paddle       0.00      0.00      0.00         0\n",
      "                       padlock       0.33      0.50      0.40         2\n",
      "                    paintbrush       0.00      0.00      0.00         1\n",
      "                        pajama       0.50      0.50      0.50         2\n",
      "                        palace       1.00      1.00      1.00         1\n",
      "                       panpipe       0.00      0.00      0.00         1\n",
      "                   paper towel       1.00      0.50      0.67         2\n",
      "                      papillon       1.00      1.00      1.00         1\n",
      "                     parachute       1.00      1.00      1.00         1\n",
      "                    park bench       0.67      1.00      0.80         2\n",
      "                 parking meter       0.00      0.00      0.00         1\n",
      "                     partridge       1.00      0.50      0.67         2\n",
      "                 passenger car       0.50      1.00      0.67         1\n",
      "                         patas       1.00      0.33      0.50         3\n",
      "                         patio       0.00      0.00      0.00         1\n",
      "                     pay-phone       1.00      1.00      1.00         1\n",
      "                       peacock       0.00      0.00      0.00         1\n",
      "                      pedestal       0.00      0.00      0.00         0\n",
      "                       pelican       1.00      1.00      1.00         1\n",
      "                    pencil box       1.00      0.50      0.67         2\n",
      "              pencil sharpener       0.00      0.00      0.00         1\n",
      "                       perfume       1.00      1.00      1.00         1\n",
      "                   photocopier       1.00      1.00      1.00         1\n",
      "                          pick       0.50      1.00      0.67         1\n",
      "                   pickelhaube       1.00      1.00      1.00         1\n",
      "                  picket fence       1.00      0.50      0.67         2\n",
      "                        pickup       0.00      0.00      0.00         1\n",
      "                          pier       0.00      0.00      0.00         1\n",
      "                    piggy bank       0.00      0.00      0.00         1\n",
      "                   pill bottle       0.00      0.00      0.00         1\n",
      "                        pillow       0.00      0.00      0.00         0\n",
      "                     pineapple       0.00      0.00      0.00         1\n",
      "                ping-pong ball       0.50      1.00      0.67         1\n",
      "                      pinwheel       1.00      1.00      1.00         1\n",
      "                        pirate       1.00      1.00      1.00         1\n",
      "                       pitcher       1.00      0.50      0.67         2\n",
      "                         pizza       1.00      1.00      1.00         1\n",
      "                         plane       0.00      0.00      0.00         1\n",
      "                   planetarium       1.00      1.00      1.00         1\n",
      "                   plastic bag       0.00      0.00      0.00         2\n",
      "                         plate       0.00      0.00      0.00         1\n",
      "                      platypus       0.50      1.00      0.67         1\n",
      "                          pole       0.00      0.00      0.00         1\n",
      "                       polecat       0.00      0.00      0.00         1\n",
      "                   pomegranate       1.00      1.00      1.00         1\n",
      "                        poncho       0.50      1.00      0.67         1\n",
      "                    pool table       1.00      1.00      1.00         1\n",
      "                    pop bottle       0.00      0.00      0.00         1\n",
      "                     porcupine       0.33      1.00      0.50         1\n",
      "                           pot       1.00      0.50      0.67         2\n",
      "                        potpie       0.00      0.00      0.00         1\n",
      "                potter's wheel       1.00      1.00      1.00         1\n",
      "                   power drill       0.00      0.00      0.00         1\n",
      "               prairie chicken       1.00      1.00      1.00         1\n",
      "                    prayer rug       0.33      1.00      0.50         1\n",
      "                       printer       1.00      0.50      0.67         2\n",
      "                        prison       1.00      1.00      1.00         1\n",
      "              proboscis monkey       1.00      1.00      1.00         1\n",
      "                     projector       0.00      0.00      0.00         0\n",
      "                    promontory       1.00      1.00      1.00         2\n",
      "                     ptarmigan       1.00      1.00      1.00         1\n",
      "                        puffer       0.00      0.00      0.00         1\n",
      "                           pug       1.00      0.50      0.67         2\n",
      "                  punching bag       0.00      0.00      0.00         1\n",
      "                         purse       0.00      0.00      0.00         2\n",
      "                         quail       1.00      1.00      1.00         1\n",
      "                         quill       0.00      0.00      0.00         2\n",
      "                         racer       0.00      0.00      0.00         1\n",
      "                        racket       1.00      1.00      1.00         1\n",
      "                      radiator       1.00      1.00      1.00         1\n",
      "                         radio       1.00      1.00      1.00         1\n",
      "               radio telescope       0.50      1.00      0.67         1\n",
      "                   rain barrel       0.00      0.00      0.00         1\n",
      "                           ram       0.00      0.00      0.00         1\n",
      "                      rapeseed       1.00      1.00      1.00         1\n",
      "          recreational vehicle       1.00      1.00      1.00         1\n",
      "                       red fox       0.33      0.50      0.40         2\n",
      "                      red wine       0.00      0.00      0.00         0\n",
      "                      red wolf       0.00      0.00      0.00         2\n",
      "          red-backed sandpiper       0.00      0.00      0.00         1\n",
      "        red-breasted merganser       1.00      1.00      1.00         1\n",
      "                       redbone       0.00      0.00      0.00         1\n",
      "                      redshank       0.50      1.00      0.67         1\n",
      "                          reel       0.00      0.00      0.00         1\n",
      "                 reflex camera       0.00      0.00      0.00         0\n",
      "                  refrigerator       0.00      0.00      0.00         0\n",
      "                    restaurant       0.00      0.00      0.00         3\n",
      "                      revolver       1.00      1.00      1.00         1\n",
      "                         rifle       0.00      0.00      0.00         2\n",
      "                       ringlet       0.50      1.00      0.67         1\n",
      "                ringneck snake       0.00      0.00      0.00         1\n",
      "                         robin       0.50      1.00      0.67         1\n",
      "                   rock beauty       0.00      0.00      0.00         0\n",
      "                     rock crab       0.00      0.00      0.00         1\n",
      "                   rock python       0.00      0.00      0.00         1\n",
      "                 rocking chair       1.00      1.00      1.00         1\n",
      "                    rotisserie       1.00      0.50      0.67         2\n",
      "                 rubber eraser       0.00      0.00      0.00         1\n",
      "               ruddy turnstone       1.00      1.00      1.00         1\n",
      "                 ruffed grouse       0.00      0.00      0.00         0\n",
      "                    rugby ball       0.00      0.00      0.00         1\n",
      "                          rule       1.00      0.50      0.67         2\n",
      "                  running shoe       0.00      0.00      0.00         1\n",
      "                          safe       0.50      0.50      0.50         2\n",
      "                    safety pin       1.00      1.00      1.00         2\n",
      "                    saltshaker       0.00      0.00      0.00         0\n",
      "                        sandal       1.00      1.00      1.00         1\n",
      "                       sandbar       1.00      1.00      1.00         1\n",
      "                        sarong       0.00      0.00      0.00         1\n",
      "                           sax       0.00      0.00      0.00         1\n",
      "                      scabbard       1.00      0.33      0.50         3\n",
      "                         scale       0.00      0.00      0.00         1\n",
      "                    schipperke       0.50      1.00      0.67         2\n",
      "                    school bus       0.50      1.00      0.67         1\n",
      "                    scoreboard       0.33      1.00      0.50         1\n",
      "                      scorpion       0.00      0.00      0.00         1\n",
      "                         screw       1.00      1.00      1.00         1\n",
      "                   scuba diver       1.00      1.00      1.00         1\n",
      "                   sea anemone       0.00      0.00      0.00         0\n",
      "                  sea cucumber       1.00      1.00      1.00         1\n",
      "                      sea lion       0.50      1.00      0.67         1\n",
      "                      sea slug       1.00      1.00      1.00         1\n",
      "                    sea urchin       0.00      0.00      0.00         1\n",
      "                     seat belt       0.25      1.00      0.40         1\n",
      "                sewing machine       0.00      0.00      0.00         1\n",
      "                        shield       0.00      0.00      0.00         0\n",
      "                     shoe shop       1.00      1.00      1.00         1\n",
      "                         shoji       1.00      1.00      1.00         1\n",
      "               shopping basket       0.50      1.00      0.67         1\n",
      "                 shopping cart       0.00      0.00      0.00         0\n",
      "                        shovel       0.00      0.00      0.00         1\n",
      "                    shower cap       0.33      1.00      0.50         1\n",
      "                shower curtain       0.50      1.00      0.67         1\n",
      "                       siamang       0.00      0.00      0.00         2\n",
      "                    sidewinder       0.00      0.00      0.00         1\n",
      "                 silky terrier       1.00      1.00      1.00         1\n",
      "                           ski       1.00      0.50      0.67         2\n",
      "                      ski mask       0.00      0.00      0.00         0\n",
      "                         skunk       0.50      1.00      0.67         2\n",
      "                  sleeping bag       0.50      1.00      0.67         1\n",
      "                    slide rule       0.00      0.00      0.00         1\n",
      "                  sliding door       0.67      1.00      0.80         2\n",
      "                          slot       0.50      1.00      0.67         1\n",
      "                    sloth bear       1.00      1.00      1.00         1\n",
      "                          slug       0.00      0.00      0.00         1\n",
      "                         snail       0.00      0.00      0.00         1\n",
      "                       snorkel       1.00      0.50      0.67         2\n",
      "                  snow leopard       1.00      1.00      1.00         1\n",
      "                    snowmobile       1.00      1.00      1.00         1\n",
      "                      snowplow       0.50      1.00      0.67         1\n",
      "                soap dispenser       0.00      0.00      0.00         3\n",
      "                   soccer ball       1.00      1.00      1.00         1\n",
      "   soft-coated wheaten terrier       1.00      0.33      0.50         3\n",
      "                    solar dish       0.00      0.00      0.00         1\n",
      "                      sombrero       0.00      0.00      0.00         1\n",
      "                        sorrel       0.50      0.50      0.50         2\n",
      "                     soup bowl       0.50      1.00      0.67         1\n",
      "              spaghetti squash       0.00      0.00      0.00         1\n",
      "                     speedboat       1.00      1.00      1.00         1\n",
      "                    spider web       1.00      1.00      1.00         1\n",
      "                       spindle       0.00      0.00      0.00         0\n",
      "                 spiny lobster       1.00      1.00      1.00         1\n",
      "                     spoonbill       1.00      1.00      1.00         1\n",
      "                    sports car       0.00      0.00      0.00         1\n",
      "                     spotlight       0.00      0.00      0.00         1\n",
      "            spotted salamander       1.00      1.00      1.00         1\n",
      "               squirrel monkey       0.00      0.00      0.00         1\n",
      "                         stage       0.25      1.00      0.40         1\n",
      "            standard schnauzer       0.00      0.00      0.00         1\n",
      "                      starfish       0.00      0.00      0.00         1\n",
      "              steam locomotive       0.50      1.00      0.67         1\n",
      "             steel arch bridge       0.00      0.00      0.00         0\n",
      "                    steel drum       0.50      1.00      0.67         1\n",
      "                   stethoscope       0.00      0.00      0.00         1\n",
      "                      stingray       0.50      1.00      0.67         1\n",
      "                     stinkhorn       0.00      0.00      0.00         1\n",
      "                         stole       0.00      0.00      0.00         1\n",
      "                    stone wall       1.00      1.00      1.00         1\n",
      "                         stove       0.50      1.00      0.67         1\n",
      "                      strainer       0.50      1.00      0.67         1\n",
      "                    strawberry       0.00      0.00      0.00         1\n",
      "                   street sign       0.00      0.00      0.00         1\n",
      "                     streetcar       0.50      1.00      0.67         1\n",
      "                     stretcher       0.00      0.00      0.00         1\n",
      "                  studio couch       1.00      1.00      1.00         1\n",
      "                         stupa       1.00      1.00      1.00         1\n",
      "                      sturgeon       0.00      0.00      0.00         1\n",
      "                     submarine       0.00      0.00      0.00         1\n",
      "                          suit       0.00      0.00      0.00         1\n",
      "             sulphur butterfly       1.00      1.00      1.00         1\n",
      "      sulphur-crested cockatoo       0.00      0.00      0.00         1\n",
      "                       sundial       0.00      0.00      0.00         0\n",
      "                      sunglass       0.00      0.00      0.00         1\n",
      "                     sunscreen       1.00      0.50      0.67         2\n",
      "             suspension bridge       0.00      0.00      0.00         1\n",
      "                          swab       0.00      0.00      0.00         0\n",
      "                    sweatshirt       0.00      0.00      0.00         0\n",
      "               swimming trunks       0.50      1.00      0.67         1\n",
      "                         swing       0.00      0.00      0.00         3\n",
      "                        switch       0.00      0.00      0.00         0\n",
      "                       syringe       0.00      0.00      0.00         1\n",
      "                         tabby       1.00      0.50      0.67         2\n",
      "                    table lamp       0.50      1.00      0.67         1\n",
      "                   tailed frog       0.50      0.50      0.50         2\n",
      "                          tank       1.00      0.50      0.67         2\n",
      "                   tape player       0.00      0.00      0.00         2\n",
      "                     tarantula       1.00      1.00      1.00         1\n",
      "                        teapot       1.00      1.00      1.00         1\n",
      "                         teddy       1.00      1.00      1.00         1\n",
      "                    television       0.00      0.00      0.00         1\n",
      "                         tench       1.00      1.00      1.00         1\n",
      "                   tennis ball       0.50      1.00      0.67         1\n",
      "                      terrapin       1.00      0.67      0.80         3\n",
      "                        thatch       0.00      0.00      0.00         1\n",
      "               theater curtain       0.50      1.00      0.67         1\n",
      "                       thimble       0.00      0.00      0.00         2\n",
      "              three-toed sloth       0.50      1.00      0.67         1\n",
      "                        throne       0.50      1.00      0.67         1\n",
      "                 thunder snake       0.00      0.00      0.00         1\n",
      "                          tick       0.00      0.00      0.00         0\n",
      "                         tiger       1.00      1.00      1.00         2\n",
      "                  tiger beetle       0.00      0.00      0.00         1\n",
      "                   tiger shark       1.00      1.00      1.00         2\n",
      "                     tile roof       1.00      1.00      1.00         1\n",
      "                   timber wolf       0.00      0.00      0.00         1\n",
      "                       toaster       0.00      0.00      0.00         0\n",
      "                   toilet seat       1.00      1.00      1.00         2\n",
      "                 toilet tissue       0.20      0.50      0.29         2\n",
      "                    totem pole       0.50      1.00      0.67         1\n",
      "                        toucan       1.00      1.00      1.00         1\n",
      "                     tow truck       0.50      1.00      0.67         1\n",
      "                    toy poodle       0.00      0.00      0.00         1\n",
      "                       toyshop       0.00      0.00      0.00         1\n",
      "                       tractor       1.00      1.00      1.00         1\n",
      "                 traffic light       1.00      0.50      0.67         2\n",
      "                 trailer truck       1.00      1.00      1.00         1\n",
      "                          tray       0.00      0.00      0.00         1\n",
      "                     tree frog       1.00      1.00      1.00         1\n",
      "                   trench coat       0.00      0.00      0.00         1\n",
      "                   triceratops       0.00      0.00      0.00         2\n",
      "                      tricycle       0.00      0.00      0.00         3\n",
      "                        trifle       1.00      1.00      1.00         1\n",
      "                     trilobite       0.50      1.00      0.67         1\n",
      "                      trimaran       1.00      1.00      1.00         1\n",
      "                        tripod       1.00      1.00      1.00         1\n",
      "                triumphal arch       0.50      1.00      0.67         1\n",
      "                    trolleybus       0.00      0.00      0.00         1\n",
      "                      trombone       0.00      0.00      0.00         1\n",
      "                     turnstile       0.00      0.00      0.00         1\n",
      "                        tusker       1.00      0.50      0.67         2\n",
      "           typewriter keyboard       1.00      0.50      0.67         2\n",
      "                      umbrella       0.50      0.50      0.50         2\n",
      "                      unicycle       0.50      1.00      0.67         1\n",
      "                       upright       0.50      1.00      0.67         1\n",
      "                        vacuum       1.00      0.50      0.67         2\n",
      "                        valley       1.00      0.50      0.67         2\n",
      "                          vase       0.00      0.00      0.00         0\n",
      "                         vault       1.00      0.33      0.50         3\n",
      "                        velvet       0.00      0.00      0.00         1\n",
      "               vending machine       1.00      1.00      1.00         1\n",
      "                       viaduct       1.00      1.00      1.00         1\n",
      "                    vine snake       0.00      0.00      0.00         1\n",
      "                        violin       1.00      1.00      1.00         1\n",
      "                        vizsla       0.00      0.00      0.00         1\n",
      "                       volcano       1.00      1.00      1.00         1\n",
      "                    volleyball       1.00      1.00      1.00         1\n",
      "                       vulture       0.00      0.00      0.00         1\n",
      "                   waffle iron       0.00      0.00      0.00         1\n",
      "                 walking stick       0.00      0.00      0.00         1\n",
      "                    wall clock       1.00      1.00      1.00         2\n",
      "                       wallaby       1.00      1.00      1.00         1\n",
      "                        wallet       0.00      0.00      0.00         0\n",
      "                      wardrobe       0.50      1.00      0.67         1\n",
      "                      warplane       0.00      0.00      0.00         0\n",
      "                       warthog       0.50      1.00      0.67         1\n",
      "                     washbasin       0.00      0.00      0.00         3\n",
      "                        washer       1.00      1.00      1.00         1\n",
      "                 water buffalo       1.00      0.50      0.67         2\n",
      "                     water jug       0.00      0.00      0.00         2\n",
      "                   water ouzel       1.00      1.00      1.00         1\n",
      "                   water snake       1.00      1.00      1.00         1\n",
      "                   water tower       1.00      1.00      1.00         2\n",
      "                        weasel       0.00      0.00      0.00         0\n",
      "                      web site       1.00      0.33      0.50         3\n",
      "                        weevil       0.00      0.00      0.00         1\n",
      "                      whiptail       0.00      0.00      0.00         2\n",
      "                   whiskey jug       1.00      1.00      1.00         1\n",
      "                       whistle       1.00      1.00      1.00         1\n",
      "                   white stork       1.00      1.00      1.00         1\n",
      "                           wig       0.00      0.00      0.00         1\n",
      "                     wild boar       0.00      0.00      0.00         1\n",
      "                 window screen       1.00      0.50      0.67         2\n",
      "                  window shade       1.00      1.00      1.00         1\n",
      "                   wine bottle       0.00      0.00      0.00         1\n",
      "                          wing       1.00      1.00      1.00         1\n",
      "       wire-haired fox terrier       1.00      1.00      1.00         1\n",
      "                   wolf spider       1.00      0.50      0.67         2\n",
      "                        wombat       0.00      0.00      0.00         1\n",
      "                   wood rabbit       0.00      0.00      0.00         0\n",
      "                  wooden spoon       0.50      0.50      0.50         2\n",
      "                          wool       0.00      0.00      0.00         2\n",
      "                    worm fence       1.00      1.00      1.00         1\n",
      "                         wreck       0.50      1.00      0.67         1\n",
      "                          yawl       1.00      1.00      1.00         1\n",
      "         yellow lady's slipper       1.00      1.00      1.00         1\n",
      "                          yurt       1.00      1.00      1.00         1\n",
      "                         zebra       0.00      0.00      0.00         1\n",
      "                      zucchini       1.00      1.00      1.00         1\n",
      "\n",
      "                      accuracy                           0.54      1000\n",
      "                     macro avg       0.47      0.52      0.47      1000\n",
      "                  weighted avg       0.53      0.54      0.51      1000\n",
      "\n"
     ]
    }
   ],
   "source": [
    "eval_key = \"class_eval\"\n",
    "    # Now evaluate on the \"defect2\" field\n",
    "eval_classif_padim = dataset.evaluate_classifications(\n",
    "    \"predictions\",\n",
    "    gt_field=\"ground_truth\",\n",
    "    method=\"simple\", #method is important to see data in the FO app\n",
    "    eval_key=eval_key,  # <--- store this run under \"classification_eval\"\n",
    ")\n",
    "eval_classif_padim.print_report()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![imagenet_evaluation](https://cdn.voxel51.com/getting_started_model_dataset_zoo/notebook2/imagenet_evaluation.webp)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "\n",
    "FiftyOne Zoo simplifies the process of working with computer vision datasets and models. It provides a valuable resource for researchers, developers, and enthusiasts.\n",
    "\n",
    "### Further Exploration:\n",
    "* Explore the [FiftyOne documentation](https://docs.voxel51.com/) for more advanced features.\n",
    "* Try different datasets and models from the Zoo.\n",
    "* Integrate FiftyOne Zoo into your computer vision workflows.\n",
    "\n",
    "## Next Steps\n",
    "\n",
    "To continue exploring, check out:\n",
    "- [Getting Started with FiftyOne](https://beta-docs.voxel51.com/getting_started/)\n",
    "- [Other Datasets](https://beta-docs.voxel51.com/data/dataset_zoo/)\n",
    "- [Other Models](https://beta-docs.voxel51.com/models/model_zoo/)\n",
    "- Join our [Discord community](https://community.voxel51.com)\n",
    "- Follow us on [LinkedIn](https://www.linkedin.com/company/voxel51/)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "env",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
